监督的多视图立体声（MVS）方法在重建质量方面取得了显着进步，但遭受了收集大规模基础真相深度的挑战。在本文中，我们提出了一种基于知识蒸馏的MVS的新型自我监督培训管道，称为\ textit {kd-Mvs}，主要由自我监督的教师培训和基于蒸馏的学生培训组成。具体而言，使用光度和特征一致性同时以自学的方式对教师模型进行了训练。然后，我们通过概率知识转移将教师模型的知识提炼为学生模型。在对经过验证的知识的监督下，学生模型能够以很大的优势优于其老师。在多个数据集上进行的广泛实验表明，我们的方法甚至可以胜过监督方法。

最近，通过神经网络参数化的隐式神经表示（INR）已成为一种强大而有前途的工具，可以代表不同种类的信号，因为其连续的，可区分的属性，表现出与经典离散表示的优越性。但是，对INR的神经网络的培训仅利用输入输出对，而目标输出相对于输入的衍生物通常忽略了输入。在本文中，我们为目标输出为图像像素的INR提出了一个训练范式，以编码图像衍生物除了神经网络中的图像值外。具体而言，我们使用有限的差异来近似图像导数。我们展示了如何利用训练范式来解决典型的INRS问题，即图像回归和逆渲染，并证明这种训练范式可以提高INR的数据效率和概括能力。我们方法的代码可在\ url {https://github.com/megvii-research/sobolev_inrs}中获得。

人类可以利用先前的经验，并从少数示威活动中学习新颖的任务。与旨在通过更好的算法设计来快速适应的离线元强化学习相反，我们研究了建筑归纳偏见对少量学习能力的影响。我们提出了一个基于及时的决策变压器（提示-DT），该变压器利用了变压器体系结构和及时框架的顺序建模能力，以在离线RL中实现少量适应。我们设计了轨迹提示，其中包含少量演示的片段，并编码特定于任务的信息以指导策略生成。我们在五个Mujoco控制基准中进行的实验表明，提示-DT是一个强大的少数学习者，而没有对看不见的目标任务进行任何额外的填充。提示-DT的表现优于其变体和强大的元线RL基线，只有一个轨迹提示符只包含少量时间段。提示-DT也很健壮，可以提示长度更改并可以推广到分布（OOD）环境。

基于学习的多视图立体声（MVS）方法取得了令人印象深刻的进步，并且近年来超越了传统方法。但是，它们的准确性和完整性仍在挣扎。在本文中，我们提出了一种新方法，以增强受对比度学习和功能匹配启发的现有网络的性能。首先，我们提出了一个对比匹配损失（CML），该损失将正确的匹配点视为正样品，将正确的匹配点视为正样本，并将其他点视为阴性样本，并根据特征的相似性计算对比度损失。我们进一步提出了一个加权局灶性损失（WFL），以提高分类能力，从而削弱了根据预测的置信度，在不重要的区域中低信任像素对损失的贡献。在DTU，坦克和寺庙和混合MVS数据集上进行的广泛实验表明，我们的方法可实现最先进的性能，并在基线网络上取得了重大改进。

在本文中，我们基于我们对多视图立体声（MVS）中的特征匹配的探索来呈现TransVSNet。我们将MVS模拟返回其特征匹配任务的性质，因此提出了一个强大的功能匹配变换器（FMT），以利用（自我）和（交叉）关注（交叉）在图像内和跨越图像中聚合的长程上下文信息。为了便于更好地调整FMT，我们利用自适应接收领域（ARF）模块，以确保在特征范围内平滑过境，并使用特征途径桥接不同阶段，以通过不同尺度的转换特征和梯度。此外，我们应用配对特征相关性以测量特征之间的相似性，并采用歧义降低焦损，以加强监管。据我们所知，TransmVSNet首次尝试将变压器利用到MV的任务。因此，我们的方法在DTU数据集，坦克和寺庙基准测试和BlendedMVS数据集中实现了最先进的性能。我们的方法代码将在https://github.com/megviirobot/transmvsnet中提供。

在本文中，我们提出了专家（COE）框架的合作，将多个网络的专业知识汇集在一起，以实现共同的目标。每个专家都是一个在数据集的独特部分方面具有专业知识的个人网络，可增强集体能力。给定样本，代表们选择了专家，该专家同时输出了一个粗略的预测以支持早期终止。为了实现这一框架，我们建议三个模块促使每个模型发挥其作用，即重量生成模块（WGM），标签生成模块（LGM）和方差计算模块（VCM）。我们的方法实现了ImageNet上最新的性能，以194m的触角为80.7％的前1位精度。结合PWLU激活函数和CORDCONV，COE首次仅用100m拖鞋就能实现80.0％的精度。更重要的是，我们的方法是硬件友好型，与某些现有的条件计算方法相比，达到了3-6倍的速度。

Optimization in multi-task learning (MTL) is more challenging than single-task learning (STL), as the gradient from different tasks can be contradictory. When tasks are related, it can be beneficial to share some parameters among them (cooperation). However, some tasks require additional parameters with expertise in a specific type of data or discrimination (specialization). To address the MTL challenge, we propose Mod-Squad, a new model that is Modularized into groups of experts (a 'Squad'). This structure allows us to formalize cooperation and specialization as the process of matching experts and tasks. We optimize this matching process during the training of a single model. Specifically, we incorporate mixture of experts (MoE) layers into a transformer model, with a new loss that incorporates the mutual dependence between tasks and experts. As a result, only a small set of experts are activated for each task. This prevents the sharing of the entire backbone model between all tasks, which strengthens the model, especially when the training set size and the number of tasks scale up. More interestingly, for each task, we can extract the small set of experts as a standalone model that maintains the same performance as the large model. Extensive experiments on the Taskonomy dataset with 13 vision tasks and the PASCAL-Context dataset with 5 vision tasks show the superiority of our approach.

Image instance segmentation is a fundamental research topic in autonomous driving, which is crucial for scene understanding and road safety. Advanced learning-based approaches often rely on the costly 2D mask annotations for training. In this paper, we present a more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), which leverages the off-the-shelf 3D data, i.e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models. Our LWSIS not only exploits the complementary information in multimodal data during training, but also significantly reduces the annotation cost of the dense 2D masks. In detail, LWSIS consists of two crucial modules, Point Label Assignment (PLA) and Graph-based Consistency Regularization (GCR). The former module aims to automatically assign the 3D point cloud as 2D point-wise labels, while the latter further refines the predictions by enforcing geometry and appearance consistency of the multimodal data. Moreover, we conduct a secondary instance segmentation annotation on the nuScenes, named nuInsSeg, to encourage further research on multimodal perception tasks. Extensive experiments on the nuInsSeg, as well as the large-scale Waymo, show that LWSIS can substantially improve existing weakly supervised segmentation models by only involving 3D data during training. Additionally, LWSIS can also be incorporated into 3D object detectors like PointPainting to boost the 3D detection performance for free. The code and dataset are available at https://github.com/Serenos/LWSIS.

Recently, Vehicle-to-Everything(V2X) cooperative perception has attracted increasing attention. Infrastructure sensors play a critical role in this research field, however, how to find the optimal placement of infrastructure sensors is rarely studied. In this paper, we investigate the problem of infrastructure sensor placement and propose a pipeline that can efficiently and effectively find optimal installation positions for infrastructure sensors in a realistic simulated environment. To better simulate and evaluate LiDAR placement, we establish a Realistic LiDAR Simulation library that can simulate the unique characteristics of different popular LiDARs and produce high-fidelity LiDAR point clouds in the CARLA simulator. Through simulating point cloud data in different LiDAR placements, we can evaluate the perception accuracy of these placements using multiple detection models. Then, we analyze the correlation between the point cloud distribution and perception accuracy by calculating the density and uniformity of regions of interest. Experiments show that the placement of infrastructure LiDAR can heavily affect the accuracy of perception. We also analyze the correlation between perception performance in the region of interest and LiDAR point cloud distribution and validate that density and uniformity can be indicators of performance.

While large pre-trained models have transformed the field of natural language processing (NLP), the high training cost and low cross-lingual availability of such models prevent the new advances from being equally shared by users across all languages, especially the less spoken ones. To promote equal opportunities for all language speakers in NLP research and to reduce energy consumption for sustainability, this study proposes an effective and energy-efficient framework GreenPLM that uses bilingual lexicons to directly translate language models of one language into other languages at (almost) no additional cost. We validate this approach in 18 languages and show that this framework is comparable to, if not better than, other heuristics trained with high cost. In addition, when given a low computational cost (2.5\%), the framework outperforms the original monolingual language models in six out of seven tested languages. We release language models in 50 languages translated from English and the source code here.